News Archive

SDSC Releases Version 3.1 of Zone SRB Data Management Middleware

Upgraded Data Management System Supports Advanced Scientific Projects

Published 04/28/2004

The San Diego Supercomputer Center (SDSC) at UC San Diego has released version 3.1 of Zone SRB, the widely-used SDSC Storage Resource Broker (SRB) middleware package. One of the most advanced and comprehensive production tools in the world for scientific data management, Zone SRB supports scalable collections from single-user to large, multi-institution collaborations. By supporting federation of distributed SRB data collections, Zone SRB allows scientific users to flexibly share data and descriptive metadata across different SRB Metadata Catalog (MCAT) installations, or zones, for "any data, anywhere, anytime" access. SDSC SRB Version 3.1 offers improved data integrity and backup, cross-zone authentication delegation, and more. Version 3.1, the user manuals, and release notes are available to the research community for download as a source distribution at http://www.sdsc.edu/DICE/SRB/.

In today's environment of explosively growing scientific data collections and rapidly evolving hardware and software, scientists urgently need data tools that can scale as their data grows. The SDSC SRB data management system provides a comprehensive, stand-alone solution for the management, collaborative sharing, publication, and preservation of distributed data collections. In addition, the SRB can also serve as middleware via a rich set of APIs available to higher-level applications and by providing a management layer on top of a wide variety of storage systems.

With more than 4,000 users at hundreds of sites, more than 130 terabytes of data in some 34 million files managed at SDSC, and hundreds of terabytes managed at other sites, the SRB client-server infrastructure is meeting the data management challenges of today's scientific research. A versatile tool, the SRB is used in projects that span scientific disciplines from astronomy and environmental sciences to molecular sciences, neurosciences, physics, and chemistry. Users include data grids, digital libraries, and archives, and the SDSC SRB is part of projects across agencies including NSF, NIH, DOE, NASA, NARA, and international efforts.

"Zone SRB provides bulk data and metadata manipulation mechanisms for improved ability to work with large collections," said Reagan Moore, SDSC Distinguished Scientist and co-director of the Data and Knowledge Systems program. "All new SRB features are developed based on requirements from user communities to meet critical production data management challenges, and based on the success of the sites using SRB technology we believe that their input is essential in developing a robust data management environment."

New Features in SRB 3.1

Capabilities added to version 3.1 include expanded checksumming capabilities for enhanced data integrity, a smart backup utility that can efficiently synchronize collections, replicating only new or changed data, new cross-zone authentication-delegation features, a number of features for improved installation and administration, and a new Trash Can system that gives users much faster interactive file deletion speeds, while also allowing recovery of deleted files.

In conjunction with the release of SRB 3.1, new versions of inQ, the popular Windows graphical user interface to SRB; Jargon, the SRB Java API; and MySRB, the web-based access tool, are being released that support the SRB 3.x protocol.

SRB users have participated in developing some of the new features in SRB 3.1. "We're very grateful that Ananta Manandhar of the Data Management Group of the Council for the Central Laboratory of the Research Councils e-Science Centre in the UK visited SDSC and integrated the ability to delegate certificates, making it possible for a client's GSI delegation certificate to be used for authentication at remote sites," said Arcot Rajasekar, director of the DAKS Data Grids Technologies group at SDSC. "This is an important step in facilitating cross-zone access for Zone SRB because it supports faster multi-zone access without requiring home-zone authentication."

Collaborations with SRB users are an important source of new capabilities for the SRB. "The depth of the SRB user community and their involvement in new features like this authentication mechanism improves the SRB much faster than our SDSC team can do alone," said Rajasekar. A growing number of sites and researchers are contributing to SRB development. For example, researchers at the University of Maryland have ported the SRB MCAT to the Informix database; MIT is participating in integrating the SRB with their DSpace academic intellectual content digital library system; and partners at the University of Texas are planning to port the MCAT metadata catalog to the Windows platform.

SDSC SRB version 3.1 is supported on a wide variety of systems. The MCAT Metadata Catalog runs on Oracle, IBM DB2, Sybase, Informix, MySQL, and Postgres. The SRB Server runs on Microsoft Windows NT, 2000,and XP, as well as most UNIX platforms including Linux, IRIX, AIX, HP Tru64, and Mac OSX, and supports data in file systems, tape stores, and databases. In addition to UNIX clients , additional APIs include C and C++ library calls, Shell commands, Perl and Python load libraries, dynamic load libraries for Windows, Open Archives Interface, WSDL, and Java . Interactive browser interfaces include the Windows graphical user interface, inQ, and the Web interface, MySRB.

The SRB team, led by Reagan Moore and Arcot Rajasekar, includes chief architect Mike Wan, senior developer Wayne Schroeder, data grid application specialist George Kremenek, SRB administrator Sheau-Yen Chen, and data grid developers Charles Cowart, Lucas Gilbert, Arun Jagatheesan, Roman Olschanowsky, Antoine de Torcy, Tim Warnock, and Bing Zhu. -Paul Tooby

News Archive

SDSC Releases Version 3.1 of Zone SRB Data Management Middleware

Upgraded Data Management System Supports Advanced Scientific Projects

Related Links